Papers with Deep learning
Dive into Deep Learning for Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | GluonNLP is a powerful new toolkit that automates the most laborious aspects of deep learning for NLP. |
| Approach: | This hands-on tutorial demonstrates how to scale unsupervised pre-training techniques with Apache MXNet and GluonNLP. |
| Outcome: | This hands-on tutorial examines the challenges of scaling these models and algorithms effectively with Apache MXNet and GluonNLP. |
NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning (2021.starsem-1)
Copied to clipboard
| Challenge: | Currently, symbolic and deep learning approaches to NLI are receiving less attention. |
| Approach: | They propose a symbolic-based inference framework that integrates symbolic reasoning and semantic formalism to solve NLI tasks. |
| Outcome: | The proposed framework improves accuracy on the NLI task and on the SICK and MED datasets. |
What Have We Achieved on Text Summarization? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text summarization have been investigated, but there are still gaps between them and human professionals. |
| Approach: | They analyze 8 major sources of errors on 10 representative summarization models manually. |
| Outcome: | Aiming to gain more understanding of summarization systems with respect to their strengths and limitations on a fine-grained syntactic and semantic level, we use 8 major sources of errors on 10 representative summarizing models. |
What Works and Doesn’t Work, A Deep Decoder for Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Deep learning has demonstrated performance advantages in a wide range of natural language processing tasks. |
| Approach: | They propose to deepen the decoder layer in a Transformer model to reduce the difficulty of deep learning. |
| Outcome: | The proposed method can deepen the model on both the encoder and decoder at the same time, resulting in a deeper model and improved performance. |
Meta Learning for Natural Language Processing: A Survey (2022.naacl-main)
Copied to clipboard
| Challenge: | Meta-learning is an emerging field in machine learning, but there is no systematic survey of these approaches in NLP. |
| Approach: | They propose to introduce meta-learning and the common approaches and summarize their work and review their work in the NLP community. |
| Outcome: | The proposed methods improve performance in many NLP tasks but are limited to domains, languages, countries, or styles. |
Towards Computationally Feasible Deep Active Learning (2022.findings-naacl)
Copied to clipboard
Akim Tsvigun, Artem Shelmanov, Gleb Kuzmin, Leonid Sanochkin, Daniil Larionov, Gleb Gusev, Manvel Avetisian, Leonid Zhukov
| Challenge: | Active learning (AL) is a technique for reducing the amount of annotation required for training machine learning models. |
| Approach: | They propose two techniques that reduce the amount of time required for AL . they use pseudo-labeling and distilled models to train a successor model . |
| Outcome: | The proposed algorithm reduces the time and computational overhead required to train an acquisition model and estimate uncertainty on instances in the unlabeled pool. |
GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large numbers of model parameters have shown impressive performance on the task of discriminating between language produced by cognitively healthy individuals and those with Alzheimer’s disease (AD). |
| Approach: | They propose to use a Transformer DL model pre-trained on general English text to combine an artificially degraded version of itself with a model that generalizes well to spontaneous conversations. |
| Outcome: | The proposed method generalizes well to spontaneous conversations and generates text with characteristics associated with AD, demonstrating the induction of dementia-related linguistic anomalies. |
Mining Evidences for Concept Stock Recommendation (N18-1)
Copied to clipboard
| Challenge: | a recent announcement of a state plan to build a new economic region has led to the rise of hundreds of stocks . concepts can be useful for investors to find out relevant concept stocks for making investment decisions . a chinese research team uses deep learning to mine evidences from large textual data . |
| Approach: | They use distributed word similarities and deep reinforcement learning to learn a strategy of topic expansion from large scale textual data. |
| Outcome: | The proposed method outperforms a baseline method on two Chinese stock market datasets. |
Don’t sweat the small stuff, classify the rest: Sample Shielding to protect text classifiers against adversarial attacks (2022.naacl-main)
Copied to clipboard
| Challenge: | Current text classifiers are subject to adversarial attacks from adversaries, typically executed using machine learning methods. |
| Approach: | They propose a novel and intuitive defense strategy called Sample Shielding that is attacker and classifier agnostic and does not require reconfiguration of the classifier or external resources. |
| Outcome: | The proposed defense is attacker and classifier agnostic and does not require reconfiguration of the classifier or external resources and is simple to implement. |
Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision (D18-1)
Copied to clipboard
| Challenge: | Indirect supervision is a promising direction to address the annotation bottleneck . end-to-end modeling with probabilistic logic is often intractable due to inference and learning . |
| Approach: | They propose a framework for indirect supervision that integrates deep learning with deep learning by combining probabilistic logic with deep-learning. |
| Outcome: | Experiments on biomedical machine reading demonstrate the potential of this framework. |
Prototype-Based Interpretability for Legal Citation Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | citation prediction is a key problem in high-stakes decision making areas such as law . experts often require interpretability for automatic systems to be utilized in practical settings . |
| Approach: | They propose to use legal citation prediction to solve a problem with legal experts' feedback . they propose to add a prototype architecture to add interpretability while adhering to legal parameters . |
| Outcome: | The proposed model performs well while adhering to decision parameters used by lawyers. |
Collective Relevance Labeling for Passage Retrieval (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve IR labels are incomplete and require computational overheads. |
| Approach: | They propose to distill knowledge for informed labeling without high computation overheads at evaluation time. |
| Outcome: | The proposed model outperforms state-of-the-art models while distilling the rankings better. |
Saliency Learning: Teaching the Model Where to Pay Attention (N19-1)
Copied to clipboard
| Challenge: | Recent work on explanation and interpretation has introduced methods to provide insights toward the model’s behaviour and predictions, but they do not improve the model's reliability. |
| Approach: | They propose to provide explanation training and ensure alignment of model’s explanation with ground truth explanation to ensure the model makes correct predictions for the right reason. |
| Outcome: | The proposed method produces more reliable predictions while delivering better results compared to traditional models. |
DialogSum: A Real-Life Scenario Dialogue Summarization Dataset (2021.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show unique challenges in dialogue summarization such as spoken terms, special discourse structures, coreferences and ellipsis, pragmatics and social common sense. |
| Approach: | They propose a large-scale labeled dialogue summarization dataset . they use state-of-the-art neural models to analyze spoken dialogue summaries . |
| Outcome: | The proposed dataset can be used to analyze spoken dialogue summarization challenges. |
Voice synthesis in Polish and English - analyzing prediction differences in speaker verification systems (2025.coling-main)
Copied to clipboard
| Challenge: | Using audio deepfakes, we can create high quality false voice recordings convincing enough to deceive human ears and pose security concerns. |
| Approach: | They examine the effects of deepfakes on speaker recognition systems across English and Polish corpora, evaluating both Text-to-Speech and Voice Conversion methods. |
| Outcome: | The proposed methods can maintain personal traits, posing risks of unauthorized access, and can be used to deceive human ears. |
MentalGLM Series: Explainable Large Language Models for Mental Health Analysis on Chinese Social Media (2025.emnlp-main)
Copied to clipboard
Wei Zhai, Nan Bai, Qing Zhao, Jianqiang Li, Fan Wang, Hongzhi Qi, Meng Jiang, Xiaoqin Wang, Bing Xiang Yang, Guanghui Fu
| Challenge: | Social media is a key platform for emotional expression, yet deep learning lacks flexibility and interpretability. |
| Approach: | They propose to use Chinese social media to train interpretable mental health instruction datasets to test models' ability to explain their decisions. |
| Outcome: | The proposed models outperform deep learning and LLMs on three mental health downstream tasks and demonstrate their potential for clinical applications. |
Unveiling Project-Specific Bias in Neural Code Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) based neural code models struggle to generalize effectively to real-world inter-project out-of-distribution data. |
| Approach: | They propose a Cond-Idf measurement to measure the relatedness of a token with a label and its project-specificness. |
| Outcome: | The proposed framework improves both inter-project OOD generalization and adversarial robustness while not sacrificing accuracy on intra-project IID data. |